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ABSTRACT 

Many  physical  experiments  give  rise  to  sets  of  curves  related  by 
the  requirement  that,  although  certain  of  the  curve  parameters  may 
vary  from  curve  to  curve,  others  should  be  the  same  for  all  of  the 
curves.   To  get  the  "best"  values  of  the  common  parameters,  one  would 
like  to  fit  all  of  the  curves  simultaneously  by  the  appropriate 
theoretical  expressions.   This  paper  deals  with  this  problem,  present- 
ing an  algorithm  for  its  solution,  in  the  case  that  the  curves  are 
straight  lines  with  common  slope  and  "best"  fit  is  defined  in  the 
uniform  (or  minimax)  sense. 
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1.       INTRODUCTION 

In  numerous   physical  problems    data  is  obtained  which  is    essentially- 
linear,    and  parameters  best   describing  the  linear  relationship  are 
desired.      In  some   situations   related  experiments   give  rise  to   sets   of 
straight-line   data  which   should  all  have  the  same   slope   although  the 
intercepts  may  vary.      For  example,    one  might  be  observing  an  exponen- 
tial  decay  process 

(1)  x(t)   =   ae"3t 

in  which  the   initial   amount,    a,    varies   from  experiment  to   experiment, 
but    6   should  be  constant.      Taking  logarithms    in    (l),   one   is   led  to  the 
straight-line   analysis   problem  with    (3   appearing  as   the   slope. 

Formally,   we   state  the  problem  as   follows.      Given   a  set   of  experi- 
mental  curves   g   ,    g   ,    ...,    g    ,    find  parameters   a    ,   a    ,    ...,    a    ,    6 

such  that  the   deviations   E. (x)   =   g.(x)    -  a.    -   Bx  are  small   in  some 

li  l 


sense.      One  obvious   approach   is  to  minimize 

N 


I      I    [E.(x    )]2 


where  the  x.'s   are  the  points   of  observation.      This   is   a  standard  least- 
J 

squares  problem,    solvable  by  the  usual  technique.      It    is   perhaps 

interesting  that    if  one   simply   computes   least-squares   lines   a.    +   $.x 

to   fit    each  g.,   then  the    3  which  minimizes    (2)    is   just    3  =    (V   g.)/N. 
i  ^     i 

1 

Any  least-squares   approach  suffers,   however,    from  the   drawback  that 
errors   at    individual  points    -  or  for  certain  curves    -  may  be   very  large. 
To   avoid  this   problem,   we   suggest   using  a  uniform  or  minimax  approxima- 
tion procedure;    i.e.,   minimization  of 

(3)  M  =  max  max    |E. (x) |     , 

i     xeX        x 


where  X  here  represents   either  a  closed,   bounded  interval   or  a  finite 
number  of  points.      A  theorem  characterizing  solutions  to  this  type  of 
problem  in  a  very   general   context  has  been   given   in   [l].      In  this 
paper  we   state  hov  the  theorem  applies  to  the  problem  at   hand  and 
discuss   a  numerical   algorithm,   based  on  the  theorem,    for  the   calcula- 
tion of  these  best  uniform  approximations. 


2.   CHARACTERIZATION  AND  UNIQUENESS 

Let  A  =  (a-,  ,  •  ••»  awj  ^)  be  a  parameter  set  defining  a  proposed 
approximation  to  g  ,  ...,  g„.   Define  an  extremal  (x,  k)  of  the  approxi- 
mation A  to  be  a  number  pair  such  that  M  =  |E  (x)|.   Let  a   =  sgn  K  (x). 
The  following  theorem  then  follows  immediately  from  Theorem  3  of  [l]. 

Theorem  1.   A  necessary  and  sufficient  condition  that  A  define  a 
best  uniform  approximation  to  g  ,  •••>  g^  is  that  there  exists  no  para- 
meter set  A'  =  (a'   a',  ....  0*   ,  3  )  such  that 


1    =    (a^,    o^,    .  ..,    a^   ,    3') 


(1+)  a  1     [a«    +    3'x]    >  0 

xk       k 

for  every  extremal    (x,    k)   of  the   approximation  A. 

This   result    is,    of  course,    completely  analogous   to  the   usual   Chebyshev 
single-function  result  that   an   approximation   can  be   improved  if  and  only 
if  some  member  of  the   approximating   family  takes  on  the   same   signs    as  the 
error  curve  does   at    its   extremal  points.      In   fact,    a  better  approximation 
may  be   constructed  by  adding  on  to  the   original   approximation   some   small 
multiple  of  any  function   satisfying  the   sign   condition.      Although  this 
"inequality"   approach   can  be  used  to   construct  best   single-function 
approximations    [2],    it    is   very  seldom  done,   presumably  because   Remez-type 
algorithms    [3]   are  better  studied  and  known  to  converge  rapidly. 
For  our  problem,   however,    Remez-type   algorithms   appear  to  be   diffi- 
cult  to   construct,    and  we  shall  here   discuss    an  algorithm  based  on 
(U). 

There  are  two   conclusions   to  be  drawn   from  Theorem  1  which   are  useful 
in  simplifying  the  procedure. 

Theorem  2.      A  necessary   and  sufficient   condition  that  A  define   a  best 
uniform  approximation  to  g   ,    . . . ,    g      is  that   one  of  the   following  two 
conditions  hold: 


(i)      For  some  index  i,    a.    +   3x  is  the  best   straight-line  approxima- 
tion to   g.,   and  for  all  J   ±  i 
1 

max    |g.(x)   -  a     -   3x|    <  max    |g   (x)    -  a     -   0x|    . 
xeX       J  3  xeX 

(ii.)      For  some  pair  of  indices   i,    J,  the  parameter  set    (ou,    a^ ,    3) 
defines   a  best   simultaneous   approximation  to  the  pair  g±,    gy    and  for 

all  k  ^  if   J 

max    |gk(x)    -  ak  -   3x|    *  max    |g.(x)    -  a.    -   0x|    =  max    |g.(x)    -  a.    -   3x 

This  theorem  may  be  verified  by   carefully  checking  the  various   possibilities 
for   (h)    and  noting  that   no  more  than  two   indices   k  can  enter   into  the 
characterization   in  a  significant   or  non-redundant    fashion.      In  the   same 
manner,   one  may  verify  the   following  important   fact. 

Theorem  3.      Although  there   are   in  general  many  parameter  sets  A 
defining  a  best   approximation,   the  parameter  3   is  unique. 

This   result    is  of  extreme   importance   for  applications;    it    is   very 
often  this  best   slope  that   one  wants  to   find.      Putting  together  Theorems 
2   and  3,   we   see  that  the  main  problem  is  then  one  of  finding  this  best 
3,    and  that   this    3  must   lie  among  those  obtained  by  constructing  single- 
function  approximations    ((i)   of  Theorem  2)   or  by   constructing  best 
approximations  to   all   function  pairs    ((ii)   of  Theorem  2).      For  the 
latter,   we  use   a  procedure  based  on    (U),   which   is  not   very   complicated 
when  only  two   functions    are   involved.      In  fact,   we  have  the   following 
simple   "alternation"   theorem. 


Theorem  h_. 
Suppose  that  the  best  simultaneous  approximation  to  the  pair  gn ,  g_  is 
not  characterized  by  condition  (i)  of  Theorem  2.   Then  there  exist  four 
points  x  ,  ,  x  ,  x  ,  x   in  X  such  that 

(a)  x1±   <   x12;  x21  <  x22  ; 

(b)  (x11,  1),  (x^,  1),  (x21,  2),  (x22,  2)  are 
extremals  of  the  approximation,  and 

(c)  the  error  signs  satisfy 

°J1  °j2   =  -1'       (J  =  X'   2) 

"ll  °21   =  -1 

(where  a..  =  sgn  E.(x..)). 

J  -L  J     J  -1- 

2        3 
Example .   Suppose  g  =  x  ,  g  =  x  and  X  =  [-1,  l].   One  can  quickly  verify 

that  the  best  approximation  is  provided  by  a  =0.65,  a  =  0,  8  =  0.325. 

Extremal  x  values  are  x   =  -1,  x^   =  0.1625,  x   =  -1,  and  x   =  +1. 

The  error  norm  M  is  0.675-   The  reader  is  invited  to  provide  his  own 

quick  sketch  to  see  that  the  sign  condition  (c)  of  Theorem  k   is  satisfied. 


3.   THE  ALGORITHM 

fv\      a   (y)  a   (x)  defined  on  X,  one  proceeds 

Given  functions  g-^x),  S2{    ' y    ""'  gN 

as  follows: 

1.  For  i  =  1,  ..-,  N>  use  the  Remez  algorithm  (available  in  most 
subroutine  libraries)  to  compute  the  best  straight-line  approximation 
C     +   3.x  to  g.(x)   on  X.      Let   p     =  max    |g   (x)   -   £.    -   S.x|. 

Ill  x  X£A 

2.  For  each   3.    (i  =  1,    .  •  •  ,    N)    found  in  Step  1,   and  for  all   j   ¥   i, 
compute  the  best   constant   approximation  Cj   to   gj   -   3^,    and  let 

=max|g.(x)-3.x-c    |.       (This   is    a  trivial   computation.      Simply 
find  r^  max    (gj(x)    -   6.x)    and  rg  =  min    (gj(x)    -   3.(x)).      Then 
c.   =    (rx   +  r2)/2  and  p±J   =    (^   -  rg)/2.  )      If  for  some   i,    p..    <   P-    for  all   J, 
then  a  best   approximation   is   constructed  by  taking  a.   =   c.    (j   *   i), 

a     =    c       and  6  =   S..      (Note:      This   is  the  situation   described  in  case    (i) 

i  i'  i 

of  Theorem  2,   where  the  best   overall   slope   3   is   just  that   obtained  for 

the  best   individual   approximation  to   g..      The  nonuniqueness   of  the   a.'s 

also   shows   up  clearly  here,    since   if  p..    <   p.    for  some   J,    a  small   change 

in   a.   can  be  made  without   affecting  the  max  error.)      If  a  best    approxi- 

J 

mat  ion  has  not  yet  been  found,  one  proceeds  to  Step  3. 

3.   One  now  examines  function  pairs.   Clearly,  for  any  pair  i,  J 

such  that  either  p..  <  p.  or  p   <  p   a  best  simultaneous  approximation 

ij    i     Jx    *) 
will  be  characterized  by  slopes  of  p.  or  g.  respectively.   Since  these 

3  values  have  already  been  shown  not  to  yield  a  best-  overall  approxima- 
tion, it  is  only  for  i,  J  pairs  satisfying  neither  of  the  above 
inequalities  that  we  now  wish  to  compute  a  best  simultaneous  approximation 


{y .    +   nx,  Y.  +  tix).   The  procedure  is  iterative.   Begin  with  initial 

J 

guesses  of  the  parameters.   We  have  used 

n(0)  =  (3.  +  3.)/2, 
i    J 

y.  0)  =  c.  +  (3.  -  3.)  (a  +  b)A,  and 
y/0)  -  C.  +  (3.  -  3.)  (a  +  b)A, 

but  a  better  choice  may  be  possible.  (Here  the  parameters  a.  b  define 

the  interval  of  approximation;  i.e.   X  =  [a,  b].   If  X  is  a  finite 

Q 

point   set    {x   }        ,  ,    then  we   assume  a  =  x.,    <   x  <    ...    <   x-   =  b.) 

^  q      q=l  12  Q 


Letting 


F^      (x)    =   gk(x)    -  Yk        -     n        x. 


compute,    for   l  =   0, 

R  =  max   {max    |E.       (x)|,   max    |E.      (x)    }      , 

x  x  x  J 

and  then   iterate.  (For  simplicity,    superscripts   indicating  the   iteration 
number  will  be  generally  omitted. ) 
A.      Determine   points 


and 


Xil    <   Xi2   < 


x..,     <   x        < 
Jl  J2 


where  E.(x)    (or  E.(x))   takes   on  the  value  ±    R    (to  within  some  preassiened 
i  J 

tolerance,    of  course),    and  let   associated  error  signs  be  an     •      The 

km 

inequalities    (k)    are   now  readily  treated  by  ad  hoc  methods.      For  each  k, 
the  list    {a      }  may   contain  no  more  than  one   sign  change.    (Otherwise  we 
would  have  a  best    individual   approximation  -  a  possibility   already  elimina- 
ted in   Step  2.)      One  therefore   checks    each   of  the  two   lists    for  whether 


8 


it    (a)    is   empty   (b)   has   no  sign  changes,  or   (c)   has  one  sign  change. 
If  both  lists  have  a  sign  ehange,   and  the  changes  are  in  opposite 
directions    (i.e.   one  ♦!  to  -1   and  the   other  -1  to  +l).   then   W  has 
no  solution.      In   fact,   this    is    just   the  situation  described  in 
Theorem  k;   we  know,   therefore,  that  the   current   values   of  Yj,   Yj,   n 
are  the  best  and  we  go  on  to  Step  It. 

Otherwise,   one  readily  constructs  a  convenient  solution  to  the 
i„e9ualities   depending  upon  the  outcome  of  these  tests  plus   the   direc- 
tion of  the  sign  change   (♦  to  -,  or  -  to  ♦)   if  one  occurs.     For  example, 
suppose  that  list  i  has  a  sign  change,   say  o.2  0.3  =  -1  with  a±2  =  +1, 

«,„+   list    i   is  empty.      Then  (M  may  be  satisfied  by  choosing 
and  suppose  that  list  j   is  "*"■' 

8'    .  -1     .I   -   (x.„  ♦  x.  J/2,   and  a\  -  0.     A  simple  scheme  taking  care 

of  all  possibilities   is   easily   designed. 

-,    *•   v, '  i"™1    +   «'*     a'    +   6   x}   to  the   inequalities 
B.      Suppose   a  solution   ic^   +  B  x.   «j  *  P  w 

(    ^   -mfly    Ie   (x)   -  e(o'  +  g'x)|    for  k  =  i,    j,    and 
has  been   found.      Let  m^U)   -  max    ji^W        ev    k  I 

let   M(e)   =  max  {m. ,   m.>-      For  some   small   positive  value  of  b,   M(e)    <   R. 
After   finding  such  an   e    (see  later  discussion),   set 

(&+i)  =     U)  +      '     ,      k=  i,  j, 

Yk  ^k  k 


^  -  n(£)  +  .31. 


andR(*+l)=M(s). 

Then  return  to  Step  A. 

k.     At  this  point  we  have  obtained  a  best  simultaneous  slope  n(i,  j) 
and  corresponding  approximation  error  R(l.  j)  for  the  pair  g,,  Sy 
Precisely  as  in  Step  2  one  now  checks  whether  the  rest  of  the  »  curves 


can  be  fitted  by  straight  lines  of  slope  n(i,  j)  and  with  errors  no 
larger  than  R(i,  j).   If  they  can,  we  have  solved  our  problem  with 
6  =  n(i,  j)  and  criterion  (ii)  of  Theorem  2  is  satisfied.   If  they  cannot, 
we  go  back  to  Step  3  and  look  at  the  next  function  pair.   Theorem  2 
guarantees  that  at  least  one  pair  will  yield  a  3  that  works. 


10 


h.      DISCUSSION 

The  algorithm  has  been  programmed  (by  Dr.  Joseph  Garber)  and  tests 
on  sets  of  simple  functions  indicate  that  it  works  well. 

In  view  of  Theorem  h,    one  would  expect  that  a  minimum  of  three 
iteration  steps  (cycles  through  Steps  3A  -  B)  would  be  required  to 
obtain  a  best  approximation  to  any  pair.   That  is,  in  general  there 
will  be  only  one  extremal  point  for  the  initial  guess  and  each  itera- 
tion step  is  not  likely  to  add  more  than  one  additional  extremal 
point.   In  tests  on  four  function  pairs,  the  actual  number  of  steps 
varied  from  a  near-minimal  5  to  a  reasonable  9«   Small  changes  in 
algorithmic  details  may  improve  this  further. 

A  key  factor  affecting  the  number  of  iteration  steps  required  is 
the  choice  of  the  adjustment  parameter  e  (Step  3B).   It  therefore 
seems  worthwhile  to  include  here  a  brief  discussion  of  how  this  para- 
meter is  chosen.   Initially,  a  guess  for  e  based  on  theory  (cf.  proof 
of  Theorem  2  in  [l])  is  computed: 
eQ  =  H/2U2, 

where  H  =  max    max  [a   +  3  x| , 
k=i,j    x 

and 

U  =  min  {R    k       +8x.|l    , 

the  minimum  being  over  all  extremals  (x.  .,k). 

If  the  test  M(e  )  <  R  is  not  satisfied,  we  then  try  e   =  2~J  e   for 
j   =  1,  2,  ...  until  for  some  j   =   J  we  find  M(e  )  <  R.  Examination  of 
the  tests  shows  an  average  J  of  about  5.   Furthermore,  an  easy  way  to 
decrease  this  is  immediately  evident.   The  larger  J  values  were  often 
associated  with  an  iteration  step  in  which  the  solution  to  (h)   was 
identical  to  that  obtained  on  the  preceding  step.   It  is  reasonable 
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that,    for  two  successive   corrections   in  the   same   direction,   the   second 
should  be  of  smaller  size;    a  test   and  adjustment   of  e      for  this 
situation  may  he  readily  incorporated  into  the  program. 

After   e.    is  obtained,   this   value   is   improved  by  computing 
M    (e  p/10)    for  p  =  1,    2,    ...,    10, 
and  choosing  as    e  that   value  of  e   p/10  which  minimizes  M.      It    is 
possible  that  the  number  of  iteration   steps    (Steps    3A  -  B)   may  be 
decreased  by  sweeping  a  finer  grid  at  this   point,   but   this  may  not 
effect   any  overall   saving  in  time.      It    seems   likely,   however,   that 
some  very   different   approach   from  that   described  here  may  lead  to   a 
really  efficient  way  of  computing  a  near-optimal   e. 


12 
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related  by  the  requirement  that,  although  certain  of  the  curve 
parameters  may  vary  from  curve  to  curve,  others  should  by  the 
same  for  all  of  the  curves.   To  get  the  ,fbest"  values  of  the 
common  parameters,  one  would  like  to  fit  all  ot  the  curves  sim- 
ultaneously by  the  appropriate  theoretical  expressions.   This 
paper  deals  with  this  problem,  presenting  an  algorithm  for  its 
solution,  in  the  case  that  the  curves  are  straight  lines  with 
common  slop  and  "best"  fit  is  defined  in  the  uniform  (or  minimax) 
sense. 
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